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Abstract 


Use  of  capability  maturity  models  has  become  commonplace  among  software  development 
organizations,  especially  defense  contractors.  Government  program  offices,  however,  have 
lagged  behind  contractors  in  implementing  their  own  process  improvement  programs.  The 
difference  in  relative  maturity  between  program  offices  and  contractors  sometimes  makes  it 
difficult  for  program  offices  to  adequately  gauge  the  state  of  their  programs.  In  2004,  the 
Office  of  the  Secretary  of  Defense  announced  the  creation  of  the  CMMI®  Acquisition  Module 
(CMMI-AM).  The  module  aids  program  offices  in  developing  a  level  of  parity  with  their 
suppliers  in  terms  of  process  maturity. 

The  first  step  in  any  process  improvement  endeavor  is  to  determine  the  baseline  state.  A 
program  office  can  undergo  an  external  appraisal,  but  generally  that  is  not  a  cost-effective 
solution  for  an  organization  that  is  still  a  novice  in  process  improvement.  For  organizations 
with  little  process  improvement  experience,  a  better  choice  is  to  begin  with  a  self-assessment. 

This  guide  provides  program  managers  with  general  information  about  the  CMMI-AM, 
details  about  the  self-assessment  technique,  and  the  questions  used  in  a  self-assessment. 

After  reading  this  guide,  program  managers  can  evaluate  whether  a  self-assessment  fits  their 
needs,  and  if  so,  conduct  one. 


CMMI  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  University. 
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1  The  CMMI  Acquisition  Module 


1.1  A  Brief  History 

Use  of  capability  maturity  models  has  become  commonplace  in  the  software  industry, 
especially  among  defense  contractors.  Beginning  with  the  Capability  Maturity  Model®  for 
Software  (SW-CMM),  and  now  continuing  with  the  Capability  Maturity  Model  Integration 
(CMMI®)  framework,  software  development  organizations  have  achieved  significant  gains  in 
their  ability  to  develop  and  deliver  systems  with  predictable  results  [Goldenson  03].  Even  a 
few  government  program  offices  have  implemented  process  improvement  programs  with 
good  results  [Capell  04,  Kotchman  02]. 1  However,  most  have  lagged  behind  their  contractors 
in  the  area  of  process  maturity.  The  difference  in  relative  maturity  frequently  makes  it 
difficult  for  program  offices  to  accurately  gauge  the  state  of  their  programs  and  communicate 
with  their  contractors,  ultimately  leading  to  unpredictable  results  for  those  programs 
[Gallagher  04].  Figure  1  depicts  the  acquirer/supplier  mismatch.  Situations  where  both 
acquirers  and  suppliers  possess  high  degrees  of  technical  and  management  skill  tend  to  yield 
the  best  results,  whereas  other  combinations  tend  to  increase  the  risk  of  failure. 

The  government  increasingly  relies  on  prime  contractors,  lead  integrators,  and  the  like,  to 
operate  with  limited  supervision.  Such  trust  in  these  parties  is  not  always  warranted.  It  is 
incumbent  upon  the  government  to  maintain  some  level  of  “smart  buyer”  capability  in  order 
to  provide  effective  program  management,  oversight,  and  stewardship  of  taxpayer  funds. 


®  Capability  Maturity  Model  and  CMMI  are  registered  in  the  U.S.  Patent  and  Trademark  Office  by 
Carnegie  Mellon  University. 

1  Additional  information  about  specific,  quantitative  results  of  process  improvement  based  on  CMMI 
models  may  be  found  at  http://www.sei.cmu.edu/cmmi/results.html. 
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Figure  1:  The  Acquirer/Supplier  Mismatch 

In  2002,  Congressional  leaders  recognized  the  need  for  the  defense  department  to  improve  its 
ability  to  manage  programs,  especially  those  with  significant  software  content.  They 
included  Section  804  in  the  National  Defense  Authorization  Act  for  Fiscal  Year  2003,  which 
requires  each  of  the  Services  and  Defense  Agencies  to  establish  software  acquisition 
improvement  programs  [PL  02].  Specifically,  Section  804  states: 

“The  Secretary  of  each  military  department  shall  establish  a  program  to 
improve  the  software  acquisition  processes  of  that  military  department. 

The  head  of  each  Defense  Agency  that  manages  a  major  defense  acquisition 
program  with  a  substantial  software  component  shall  establish  a  program  to 
improve  the  software  acquisition  processes  of  that  Defense  Agency.” 

Clearly,  there  is  both  a  need  and  an  imperative  to  improve  the  government’s  ability  to 
successfully  acquire  systems  that  have  high  software  content. 


1.2  The  Acquisition  Module 

To  help  Department  of  Defense  program  offices  improve  their  abilities,  the  Office  of  the 
Secretary  of  Defense  (OSD)  announced  the  creation  of  the  CMMI  Acquisition  Module 
(CMMI-AM)  in  2004  [Bernard  04],  The  module,  which  draws  practices  from  the  CMMI 
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framework  in  addition  to  other  relevant  models.2  was  developed  to  aid  program  offices  in 
developing  a  level  of  parity  with  their  suppliers  in  terms  of  process  maturity. 

It  is  important  to  distinguish  between  CMMI  models  and  modules.  In  general.  CMMI  models 
are  the  official  documents  defining  best  practices  for  a  given  discipline.  Organizations  can 
use  models  to  achieve  a  maturity  level  rating.  CMMI  modules  are  excerpts  from  the  model, 
often  with  additional  material  provided  on  a  trial  basis.  Organizations  can  use  modules  to 
identify  their  strengths  and  weaknesses,  but  cannot  base  a  maturity  level  rating  on  them 
[Gallagher  04].  The  CMMI-AM  is  a  module. 

For  the  CMMI-AM,  selected  practices  were  extracted  from  CMMI-SE/SW/IPPD/SS  Version 
1.1  [SEI 02]  and  other  source  models  to  support  acquisition  organizations  as  they  plan 
projects,  prepare  and  execute  solicitations,  monitor  and  control  suppliers,  and  manage 
programs.  In  general,  the  CMMI-AM  uses  the  terminology  of  the  source  models,  with 
acquisition-oriented  amplification  text  added  to  help  acquirers  interpret  the  meaning  of  the 
process  areas  in  the  acquisition  context.  These  practices  provide  a  basis  for  discipline  and 
rigor,  allowing  the  acquisition  process  to  be  executed  with  repeated  success  [Bernard  04]. 


1 .3  Improvement  Via  the  Acquisition  Module 

Introduction  of  the  CMMI-AM  raises  a  very  important  question:  How  can  a  program  office 
best  make  use  of  it? 

Process  improvement  using  the  CMMI  framework  as  a  guide  entails  a  significant 
commitment  of  resources  and  time.  For  program  offices  where  process  improvement  may 
not  have  been  a  priority  in  the  past,  undertaking  a  serious  process  improvement  effort  can  be 
daunting.  The  structure  of  the  CMMI  has  been  developed  to  allow  an  organization  to  select 
areas  for  improvement  based  upon  business  needs.  Rather  than  investing  in  process 
improvement  aimed  at  a  specific  group  of  processes,  subsections  of  the  model  can  be  selected 
to  support  improvement  in  those  areas  of  the  business  that  require  immediate  attention.  The 
next  step  is  to  determine  what  those  ‘immediate  attention’  areas  might  be  for  an  organization. 
For  acquisition  program  offices,  this  is  where  the  CMMI-AM  is  most  useful.  The  CMMI- 
AM,  in  effect,  establishes  a  “starter  set”  of  process  areas  that  are  relevant  to  acquisition. 

Defining  a  set  of  high-priority  process  areas  is  only  the  beginning,  however.  The  next  step  is 
to  determine  where  an  organization  stands  with  respect  to  those  process  areas.  To  accomplish 
this  task,  there  are  two  main  choices.  One  is  to  use  the  Standard  CMMI  Appraisal  Method 
for  Process  Improvement  (SCAMPIsm),  which  provides  a  standardized  approach  for 


2  The  other  relevant  models  include  the  Software  Acquisition  Capability  Maturity  Model  (SA-CMM) 
framework  [Cooper  02]  and  the  Federal  Aviation  Administration  (FAA)  Integrated  Capability 
Maturity  Model  (FAA-iCMM)  [FAA  01]. 

SM  SCAMPI  is  a  service  mark  of  Carnegie  Mellon  University. 
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determining  process  performance.3  The  other  is  to  perform  a  self-assessment.  For  a  program 
office  with  limited  process  improvement  experience  and  resources,  this  may  be  a  more 
suitable  first  step. 

Certainly,  acquisition  organizations  can  elect  to  pursue  a  SCAMPI  Class  A  appraisal  to  gain  a 
first  insight  into  their  process  maturity,  but  that  is  generally  not  a  cost-effective  solution  for 
any  organization  that  is  initiating  a  formal  process  improvement  effort.  Self-assessment  may 
be  a  better  choice  for  these  organizations  because  using  a  SCAMPI  technique  requires  some 
understanding  of  process  improvement  and  the  CMMI  framework.  Additionally,  the 
SCAMPI  appraisal  method  requires  the  participation  of  staff  that  have  been  formally 
educated  and  authorized  to  support  the  execution  of  the  method.  In  environments  where  the 
requisite  level  of  understanding  and  training  have  not  yet  been  reached,  even  the  less  rigorous 
SCAMPI  B  and  C  methods  may  be  inappropriate  as  a  first  step  (although  SCAMPI  B  or  C 
would  be  appropriate  next  steps  after  an  organization  has  achieved  some  level  of 
improvement  following  a  self-assessment). 


3  There  are  three  classes  of  SCAMPI:  “A,”  “B,”  and  “C,”  with  A  being  the  most  rigorous,  and  the 
only  one  that  can  result  in  a  rating  relative  to  CMMI  maturity  level. 
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2  Self-Assessment  Based  on  the  CMMI-AM 


2.1  Description 

In  an  organization  where  process  maturity  is  a  new  concept,  a  self-assessment  offers  an  easy 
entree  to  the  world  of  process  improvement.  As  the  term  implies,  self-assessment  is  a  means 
by  which  an  organization  assesses  compliance  to  a  selected  reference  model  or  module 
without  requiring  a  formal  method.  Self-assessment  helps  organizations  find  gaps  between 
their  current  practices  and  the  practices  identified  in  the  CMMI-AM.  This  early  gap 
identification  allows  program  offices  to  begin  improving  their  business  practices  before 
exposing  themselves  to  the  external  scrutiny  of  a  SCAMPI  evaluation.  The  results  of  the 
self-assessment  also  can  be  used  to  educate  the  organization  about  the  acquisition  module  as 
well  as  about  the  requirements  of  the  formal  appraisal  method. 

The  mechanics  of  a  self-assessment  are  simple.  Using  a  survey,  acquisition  office  personnel 
respond  to  a  series  of  questions  based  on  their  understanding  of  how  work  is  performed  in 
their  organization.  To  encourage  candor  in  the  responses,  program  offices  should  administer 
the  survey  confidentially.  The  individual  responses  are  then  aggregated,  averaged,  and 
presented  to  the  program  office  staff  for  discussion  and  further  action. 

Figure  2  helps  illustrate  these  points.  It  shows  examples  of  the  types  of  statements  to  which 
an  organization  responds  in  a  CMMI-AM  self-assessment.  A  full  assessment  would  have 
many  more  questions  covering  all  the  process  areas  described  in  the  CMMI-AM,  as  outlined 
in  Appendix  B. 

The  statements  are  deliberately  devoid  of  process  model  terminology;  instead,  they  use 
language  that  should  be  more  familiar  and  accessible  to  program  office  personnel. 
Respondents  score  each  statement  from  1  to  10,  where  1  represents  the  statement  on  the  left 
and  10  represents  the  statement  on  the  right. 

Within  a  program  office,  key  personnel  respond  to  the  statements  based  on  their  own  point  of 
view.  Key  personnel  includes,  for  example,  the  program  manager  and  deputy  program 
manager,  the  chief  engineer,  the  chief  software  engineer,  the  contracts  specialist,  the  business 
manager,  and  the  leads  of  integrated  product  teams  (IPTs).  The  goal  is  to  get  the  widest 
response  possible  to  avoid  skewing  results. 
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1.  Estimates  are  based  on  wild  guesses  Estimates  of  project  planning 

or  dictated  from  above.  parameters  (i.e,  scope,  task 

attributes,  lifecycle,  cost,  effort,  etc.) 
are  established  and  maintained. 

|  1  2  3  4  5  6  7  8  9  10) 


2.  Plans  are  rarely  written  down  nor  do  A  project  plan  is  established  and 
they  reflect  current  project  activities.  maintained  as  the  basis  for 

managing  the  project. 


1  23456789  10 


3.  We  rarely  seek  commitments  from 
those  affected  by  the  project  plan. 

Commitments  to  the  project  plan  are 
established  and  maintained. 

|  1  2  3  4  5 

6  7  8  9  10  I 

4.  We  track  progress  based  on 
personality  and  an  arbitrary 
baseline. 

Actual  performance  and  progress  of 
the  project  are  monitored  against  the 

project  plan. 

|  1  2  3  4  5 

6  7  8  9  10  | 

5.  It  is  difficult  to  know  when  the 
project  has  deviated  from  the  plan 
based  on  the  data  we  review. 

Corrective  actions  are  managed  to 
closure  when  the  project’s 
performance  or  results  deviate 
significantly  from  the  plan. 

[  1  2  3  4  5 

6  7  8  9  10  | 

Figure  2:  Example  CMMI-AM  Self-Assessment  Questions4 


Figure  3  depicts  a  graphical  example  of  how  self-assessment  results  might  be  aggregated  for 
further  study  and  discussion  within  an  organization.  In  this  example,  a  fictitious  organization 
assessed  itself  against  the  project  management  process  areas  described  in  the  CMMI-AM. 
The  horizontal  axis  shows  the  individual  process  areas,  while  the  vertical  axis  shows  the 


4  Excerpted  from  the  SEI  white  paper  “CMMI-AM:  Goal  Implementation  Survey”  by  Brian  P. 
Gallagher.  The  full  list  of  questions  appears  in  Appendix  B  of  this  document. 
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scores.  The  bars  depict  the  range  of  scores  for  each  process  area.  Mean  scores  are  denoted 
by  the  boxes. 


Figure  3:  Example  CMMI-AM  Self-Assessment  Scoring  Sheef 

In  this  example,  the  fictitious  organization  rated  itself  on  the  low  side  of  average  overall,  as 
determined  by  the  mean  scores  (all  less  than  5).  The  organization  rated  two  process  areas, 
Establish  Estimates  and  Select  Suppliers,  low  (mean  scores  just  below  3.5).  One  process 
area,  Manage  Corrective  Action  to  Closure,  received  scores  as  low  as  1  and  as  high  as  10, 
with  a  mean  score  of  just  under  5.  All  of  the  process  areas  indicate  a  wide  range  of 
responses. 

The  fictitious  acquisition  office  can  now  use  the  scoring  sheet  to  open  a  dialogue  about 
process  implementation  in  the  organization.  They  can  investigate  the  disparity  of  responses 
among  the  staff  and  discuss  what  needs  to  be  done  to  get  a  consistent  set  of  responses  (i.e., 
why  is  it  that  someone  in  this  program  office  thinks  that  corrective  actions  are  not  managed 
to  closure  and  another  person  believes  that  the  program  office  uses  a  rigorous  method  to 
manage  corrective  actions  to  closure?).  After  the  disparity  in  responses  is  addressed,  the 
program  office  can  use  the  data  from  the  self-assessment  to  discuss  what  needs  to  be  done  to 
raise  the  average  response  (i.e.,  what  does  the  program  office  need  to  do  to  establish  and 
maintain  a  more  rigorous  method  to  track  corrective  actions  to  closure?).  When  the  average 
response  for  each  process  area  is  near  10  and  the  range  of  responses  is  smaller,  the  program 

5  Excerpted  from  the  companion  tool  to  the  white  paper  “CMMI-AM:  Goal  Implementation  Survey” 
by  Brian  P.  Gallagher. 
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office  may  be  ready  for  a  SCAMPI  appraisal.  Self-assessments  tend  to  be  optimistic,  so 
following  up  with  a  SCAMPI  appraisal  after  some  initial  improvements  have  been  made  is  a 
good  way  to  hone  processes  based  on  objective  insights. 


2.2  Benefits 

Self-assessments  do  not  impact  the  daily  routine  of  a  program  office  significantly;  they  do 
not  require  the  organization  to  accommodate  a  site  visit  by  an  external  assessment  team. 
Typically,  a  SCAMPI  A  appraisal  requires  on-site  interviews  to  confirm  implementation  and 
use  of  documented  processes.  This  type  of  activity  may  require  multiple  visits  over  a  period 
of  weeks,  depending  upon  the  size  and  complexity  of  the  program  office.  Self-assessments 
still  require  that  program  office  staff  take  time  to  answer  the  questions,  but  this  is  generally 
substantially  less  effort  than  that  required  for  any  independent  appraisal  like  a  SCAMPI. 

Self-assessments  do  not  require  documentation  as  evidence  of  compliance  with  a  reference 
model,  although  having  documentation  can  be  invaluable  for  analysis  of  results,  helping  to 
answer  questions  like  “How  do  we  know?”  The  SCAMPI  methods  all  require  direct  artifacts 
of  implementation  for  each  practice  within  the  reference  model  or  module.  Self-assessments 
in  fact,  do  not  require  any  evidence  at  all.  Generally,  because  of  the  lack  of  formality  of  the 
self-assessment,  they  tend  to  be  less  expensive  to  the  program  office. 

The  general  characteristics  of  self-assessment  in  contrast  to  the  three  classes  of  SCAMPI 
appraisal  are  shown  in  Table  1.  These  characteristics  are  shown  to  provide  a  very  high-level 
view  of  the  impact  of  appraisals.  One  can  easily  see  why  self-assessment  is  an  attractive 
alternative  for  beginning  a  process  improvement  effort.  The  increasing  rigor  of  the  SCAMPI 
methods  offer  better,  and  more  objective,  visibility  into  a  program  office’s  operation, 
providing  the  opportunity  to  fine-tune  processes.  The  combination  of  techniques  provides  a 
means  for  program  offices  to  bootstrap  their  improvement  efforts  progressively. 
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Table  1:  Characteristics  of  Different  Appraisal  Methods6 


\  Methods 

SCAMPI 

Characteristics  \ 

Self- 

Assessment 

Class  C 

Class  B 

Class  A 

Amount  of  Objective 
Evidence 

(Required 

Documentation) 

None 

Low 

Medium 

High 

Ratings  Generated 

No 

No 

No 

Yes* 

(*but  not 

for  the 

CMMI- 

AM  alone) 

Resource  Needs 

Low 

Low 

Medium 

High 

Team  Size 

None 

Small 

Medium 

High 

2.3  Pitfalls 

Although  self-assessments  can  be  a  low-impact,  low-cost  way  of  gaining  insight  into  an 
organization’s  process  maturity,  they  are  not  without  shortcomings. 

For  one  thing,  there  is  the  tendency  of  people  in  an  organization  to  rate  themselves  higher 
than  an  external,  objective  appraisal  team.  Such  over-rating  can  happen  for  a  variety  of 
reasons.  In  an  organization  that  is  somewhat  unfamiliar  with  process  improvement  or 
maturity  models,  there  is  plenty  of  room  for  interpretation  of  the  questions.  It  is  not 
surprising  that  people  make  their  best  guess  when  faced  with  questions  about  an  unfamiliar 
subject.  Sometimes  people  miss  the  point  entirely,  and  instead  of  trying  to  provide  an  honest 
evaluation,  they  try  to  guess  the  “right”  answer.  Sometimes  people  provide  the  answer  that 
they  think  the  boss  wants  to  hear.  These  are  all  very  common  (and  human)  responses  when 
an  organization  embarks  on  a  path  to  affecting  change. 


6  This  table  has  been  adapted  from  the  one  found  in  Appraisal  Requirements  for  CMMl,  Version  1.1 
(ARC,  VI. 1)  [SEI 01]. 
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The  self-assessment  process  does  not  require  documentation  to  “prove”  that  business 
practices  have  been  implemented  for  the  organization.  In  addition,  no  evidence  that  shows 
the  execution  of  the  documented  practices  is  required  to  answer  the  questions  on  the  self- 
assessment.  If  the  organization  lacks  process  documents,  a  self-assessment  may  not  uncover 
the  shortfall.  This  lack  of  documentation  makes  it  difficult  to  later  demonstrate  repeatability 
of  the  business  practices  in  a  formal  manner.  More  importantly,  documented  processes 
provide  the  basis  for  uniform  understanding  and  execution  of  an  organization’s  business. 

External  auditing  organizations,  such  as  the  Government  Accountability  Office  (GAO), 
generally  do  not  regard  self-assessment  results  as  meaningful  because  of  the  informality  and 
subjective  nature  of  self-assessments.  To  achieve  credibility,  techniques  such  as  the  SCAMPI 
are  still  needed. 

Finally,  in  some  cases,  it  might  be  difficult  to  really  know  what  the  results  mean.  Do 
generally  negative  results  indicate  widespread  process  problems,  a  failure  to  communicate 
effectively  within  the  organization  about  processes,  or  a  simple  misunderstanding  of  the  self- 
assessment  questionnaire?  Likewise,  do  favorable  results  mean  the  organization  is  doing 
well,  or  do  they  indicate  people  are  guessing  at  what  they  believe  the  desired  answer  to  be?  A 
self-assessment  cannot  answer  these  questions.  Only  a  trained  appraisal  team  can  help 
answer  such  questions.  This  limitation  does  not  invalidate  self-assessment  results;  rather  it 
supports  the  CMMI  product  suite  approach  of  building  upon  the  results  of  various  gap 
analysis  and  triage  techniques.  The  self-assessment  tool  can  be  used  as  an  initial  triage 
technique,  but  it  must  be  considered  with  appropriate  cynicism.  More  formal  training,  and 
the  employment  of  appraisal  techniques  like  the  SCAMPI  B  or  C,  should  follow  the  self- 
assessments. 
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3  Summary 


This  document  provided  background  and  high-level  information  about  two  starting  points  for 
process  improvement  in  acquisition  program  offices.  The  CMMI-AM  provides  the  “starter 
set”  of  best  practices  that  acquisition  offices  can  use  to  guide  their  improvement  efforts.  Self- 
assessments  provide  a  relatively  easy  way  to  begin  the  training  and  awareness  work  that  is 
critical  to  the  success  of  an  improvement  effort,  especially  for  program  offices  that  are  just 
getting  started  in  process  improvement. 

It  has  been  said  that  a  ‘journey  of  a  thousand  miles  begins  with  the  first  step.’  Self- 
assessments  based  on  the  CMMI-AM  support  process  improvement  initiatives  in  acquisition 
program  offices  and  are  a  first  step  in  the  right  direction. 
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Feedback 


Through  its  Acquisition  Support  Program  (ASP),  the  Carnegie  Mellon®  Software 
Engineering  Institute  (SEI)  is  working  to  help  improve  the  acquisition  of  software-intensive 
systems  across  the  U.S.  government.  As  part  of  its  mission,  the  SEI  is  pleased  to  discuss  the 
information  in  this  report  in  more  detail.  The  authors  also  welcome  comments: 

Stephen  Blanchette,  Jr.  (sblanche@sei.cmu.edu) 

Kristi  L.  Keeler  (kkeeler@sei.cmu.edu) 

The  SEI  has  tools  available  to  help  program  offices  employ  CMMI-AM  based  self- 
assessments.  For  more  information  about  tools  for  self-assessment,  contact  Stephen 
Blanchette,  Jr.  at  the  email  address  above. 

For  more  information  about  the  CMMI-AM  in  general,  contact  ASP  Director,  Brian 
Gallagher  (bg@sei.cmu.edu). 


®  Carnegie  Mellon  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon 
University. 
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Appendix  A  Acronyms  and  Abbreviations 


The  alphabetical  list  below  contains  all  acronyms,  abbreviations,  and  their  meanings  as  used 
in  this  report. 


ARC 

ASP 

CMMI 

CMMI-AM 

CMMI- 

SE/SW/IPPD/SS 

CMU 

DoD 

ESC 

FAA 

GAO 

iCMM 

IPPD 

IPT 

OSD 

SA-CMM 

SCAMPI 

SEI 

SR 

SW-CMM 


Assessment  Requirements  for  CMMI 
Acquisition  Support  Program 
Capability  Maturity  Mode!  Integration 
CMMI  Acquisition  Module 

CMMI  for  System  Engineering/Software  Engineering/Integrated 
Product  and  Process  Development/Supplier  Sourcing 

Carnegie  Mellon  University 

Department  of  Defense 

Electronic  Systems  Center 

Federal  Aviation  Administration 

Government  Accountability  Office 

Integrated  Capability  Maturity  Model 

Integrated  Product  and  Process  Development 

Integrated  Product  Team 

Office  of  the  Secretary  of  Defense 

Software  Acquisition  Capability  Maturity  Model 

Standard  CMMI  Appraisal  Method  for  Process  Improvement 

Software  Engineering  Institute 

Special  Report 

Capability  Maturity  Model  for  Software 
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TN 

Technical  Note 

TR 

Technical  Report 
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Appendix  B  CMMI-AM  Evaluation  Statements 


The  following  list  is  the  full  list  of  goal  implementation  survey  questions7  for  CMMI-AM, 
version  1.0. 


Estimates  are  based  on  wild  guesses  or 
dictated  from  above. 


Estimates  of  project  planning 
parameters  (i.e..  scope,  task  attributes, 
lifecycle,  cost,  effort,  etc.)  are 
established  and  maintained. 


Plans  are  rarely  written  down  nor  do 
they  reflect  current  project  activities. 


A  project  plan  is  established  and 
maintained  as  the  basis  for  managing 
the  project. 


We  rarely  seek  commitments  from 
those  affected  by  the  project  plan. 


Commitments  to  the  project  plan  are 
established  and  maintained. 


We  track  progress  based  on  personality 
and  an  arbitrary  baseline. 


Actual  performance  and  progress  of  the 
project  are  monitored  against  the 
project  plan. 


7  Excerpted  from  the  SEI  white  paper  “CMMI-AM:  Goal  Implementation  Survey”  by  Brian  P. 
Gallagher. 
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li  is  difficult  to  know  when  the  project 

has  deviated  from  the  plan  based  on  the 

data  we  review. 

1  2  3  4 

5 

Corrective  actions  are  managed  to 

closure  when  the  project's  performance 

or  results  deviate  significantly  from  the 

plan. 

6  7  8  9  10 

There  are  no  organizational  assets 

The  project  is  conducted  using  a 

available  to  assist  in  conducting  the 

defined  process  that  is  tailored  from  the 

project. 

organization's  set  of  standard 

processes. 

1  1  2  3  4 

5 

6  7  8  9  10 

7. 

Relevant  stakeholders  for  our  project 

are  avoided  or  unknown. 

Coordination  and  collaboration  of  the 

project  with  relevant  stakeholders  are 

conducted. 

1  1  2  3  4 

5 

6 

7  8  9  10 

Project  team  members  do  not  share  a 

The  project  is  conducted  using  the 

common  vision  of  success. 

project’s  shared  vision. 

12  3  4 

5 

6 

7  8  9  10  I 

Our  integrated  teams  are  ad  hoc  and  ill- 

The  integrated  teams  needed  to  execute 

defined. 

the  project  are  identified,  defined. 

structured,  and  tasked. 

12  3  4 

5 

6 

7  8  9  10 
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16, 


Our  suppliers  are  selected  based  on 
political  whims. 


Suppliers  are  selected  based  on  the 
solicitation  package. 


1  2,3  4  5  6  7  8  9  10 


17,  Our  contracts  do  not  provide  for  the  Contracts  are  issued  based  on  the  needs 

tasks,  deliverables,  and  insight  to  meet  of  the  acquisition  and  the  suppliers’ 

our  needs.  proposed  approaches. 

I  1  ’  2  3  4  5  6  7  8  9  10 


18.  We  get  little  or  no  insight  into  the 

processes  used  by  our  suppliers  or  their 
interim  work  products. 

12  3  4 


Work  is  coordinated  with  suppliers  to 
ensure  the  contract  is  executed 
properly. 


8 


10 


19.  Our  project  team  has  a  hard  time 
knowing  what  the  requirements 
baseline  really  is. 

1  2  3 


Requirements  are  managed  and 
inconsistencies  with  project  plans  and 
work  products  are  identified. 


8 


10 


20.  Our  set  of  requirements  for  this  project 
do  not  reflect  the  needs  or  expectations 
of  the  project's  stakeholders. 


Stakeholder  needs,  expectations, 
constraints,  and  interfaces  are  collected 
and  translated  into  customer 
requirements. 


8 


10 


2 1 .  Our  requirements  are  at  such  a  high 
level,  you  could  drive  a  truck  through 
them. 

12  3  4 


Customer  requirements  are  refined  and 
elaborated  to  develop  product  and 
product  component  requirements. 


8 


10 
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28. 


I  never  know  what  is  a  controlled  work 
product  or  one  that  is  uncontrolled. 


Baselines  of  identified  work  products 
are  established. 


Changes  seem  to  creep  into  work 
products  without  my  knowledge. 


Changes  to  the  work  products  under 
configuration  management  are  tracked 
and  controlled. 


We  have  lots  of  work  products  in  the 
project,  but  1  can  never  tell  if  1  have  the 
right  version. 


Integrity  of  baselines  is  established  and 
maintained. 


No  one  cares  if  1  follow  a  process  or 
comply  with  a  template. 


1  2  3  4  5 

32.  Noncompliance  issues  are  ignored  - 

let's  face  it,  no  one  follows  the  process. 

I  1  2  3  4  5 


Adherence  of  the  performed  process 
and  associated  work  products  and 
services  to  applicable  process 
descriptions,  standards,  and  procedures 
is  objectively  evaluated. 


Noncompliance  issues  are  objectively 
tracked  and  communicated,  and 
resolution  is  ensured. 


We  collect  all  kinds  of  data  for  no 
apparent  reason. 


Measurement  objectives  and  activities 
are  aligned  with  identified  information 
needs  and  objectives. 


22 


CMU/SEI-2005-TN-004 


34. 


35. 


36. 


37. 


38. 


39. 


I  don't  get  the  data  I  need,  and  when  1  Measurement  results  that  address 

do  get  data.  I  don't  believe  it.  identified  information  needs  and 

objectives  are  provided. 


123456789  10 


The  boss  makes  all  the  decisions. 

12  3  4 

5 

6 

_  1 

Decisions  are  based  on  an  evaluation  of 

alternatives  using  established  criteria. 

7  8  9  10 

We  ignore  operational  issues  and 

Preparation  for  transition  to  operations 

logistics  -  it  just  slows  us  down  to 

and  support  is  conducted. 

listen  to  those  guys. 

12  3  4 

5 

6 

7  8  9  10 

We  don’t  have  time  to  think  about 

Acquired  products  are  transitioned  to 

transition  criteria,  we  need  to  get  our 

operations  and  support  based  on 

products  to  the  field. 

transition  criteria. 

12  3  4 

5 

6 

7  8  9  10 

We  have  insufficient  collaboration 

An  infrastructure  that  maximizes  the 

tools  and  are  not  encouraged  by 

productivity  of  people  and  affects  the 

organizational  policy  to  play  well  with 

collaboration  necessary  for  integration 

others. 

is  provided. 

12  3  4 

5 

6 

7  8  9  10 

In  our  1PT  environment,  1  can’t  tell 

People  are  managed  to  nurture  the 

who’s  in  charge,  who  1  report  to,  or 

integrative  and  collaborative  behaviors 

how  my  boss  knows  1  do  good  work. 

of  an  IPPD  environment. 

12  3  4 

5 

6 

7  8  9  10 
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